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Summary 

A technique is presented for directly encoding incoherent radiance 
fields as Gabor elementary signals. This technique uses an electro- 
acoustic sensor to modulate the electronic charges induced by the incident 
radiance field with the electric fields generated by Gaussian modulated 
sinusoidal acoustic waves. The resultant signal carries the amplitude and 
phase information required for localizing spatial frequencies of the 
radiance field. These localized spatial frequency representations provide 
a link between the either geometric or Fourier transform representations 
currently used in computer vision and pattern recognition. 
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1. Introduction 

Gabor* >2 introduced time-frequency diagrams to analyze communication 
systems. These diagrams represent the realizable tradeoff involved in 
localizing the frequency of a signal: increasing the time duration to 
improve the accuracy of a frequency measurement would also reduce the 
accuracy of the time interval determination. To optimize this basic 
tradeoff, he used Heisenberg's uncertainty principle to postulate a set of 
elementary signals which have the property that they are maximally 
localized in time and frequency. 

In treating an analogous problem concerned with the thermodynamics of 
quantum mechanical systems, Wigner^ introduced a "probability" function of 
the simultaneous values of the coordinates and momenta. This function is 
usually called "Wigner distribution function" (WDF) or, in radar 
processing, ambiguity function. When applied to communication systems, the 
WDF represents, like the Gabor diagrams, information in terms of localized 
spatial frequency spectrums. Bastiaans^ first introduced this function 
into optics to establish a link between geometric and Fourier analyses of 
optical systems. 

Bartel t et al^ developed optical methods for measuring and displaying 
time- frequency representations of one-dimensional signals, and used these 
methods to investigate sound patterns. To introduce their approach, they 
cited the musical score, which records log frequency versus time, as a 
familiar example of time-frequency representations. Bamler and 
Glunder?»8 first produced localized spatial frequency representations of 
two-dimensional targets, using coherent optical processing of photographic 
film transparencies. 
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We demonstrate analytically that it is also possible to directly 
encode incoherent radiance fields as elementary signals with an electro- 
acoustic sensor. The basic approach is to modulate the electronic charges 
induced by the incident radiance field with the electric fields generated 
by Gaussian modulated sinusoidal acoustic waves. Our analysis shows that 
the amplitude and phase information required for constructing a spatial 
location- spatial frequency representation of the radiance field can be 
extracted from the sum frequency term of the resultant signal. 

It seems reasonable to speculate that spatial location-spatial 
frequency representations of scenes may become a useful tool for computer 
vision. For example, Marcel j a 9 and Sakitt and Barlow 10 recently have 
proposed models, based upon Gabor's elementary signals, for the economical 
encoding of visual information in the cerebral cortex as depicted in 
Fig. 1. At present, computer vision tends to rely mostly on generic models 
(such as geometric shapes and allowable relationships between objects) to 
determine scene characteristics and recognize spatial patterns. The 
characterization and recognition of targets also is often performed in the 
Fourier transform domain, using coherent optical processing. The 
acquisition and reconstruction of elementary signals could establish a link 
between these two approaches to pattern recognition. 

2. Gabor Representation 

In the Gabor representation, the (one-dimensional) function F(t) is 
expanded in terms of the elementary signals 
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^ “ 9s^» ^n^ + ^9a^t; t^, f n ), (1) 

where g s (t) and g a (t) are the symmetrical and anti symmetrical elements, 
respectively, given by 


g s (t; t^,, f n ) = (2Tra 2 ) _1 / 4 e“^ t " t m^ 2 / 4 am 2 cos 2Trf n ( t-t,,,) (2a) 

g a (t; tm> f^ = (2pa 2 )-l/ 4 e“^ t " t m^ 2 / 4 am 2 sin 27rf n (t-t m ) (2b) 

and illustrated in Fig. 2. The signals are centered at the time t = 
and the frequency f = f n . 

For a signal <|»{t) with Fourier transform $(f), one can define the 
effective spread in time as 

At = [2iT(t - t) 2 ] 1 / 2 , (3a) 

and the effective spread in frequency as 


Af = [2ir(f - f) 2 ]!/2 


(3b) 


where the bar indicates an average weighted by jip( t) j 2 or |!j>(f)| 2 . The 
Gabor elementary signals then satisfy AtAf = 1/2, while for any other 
function AtAf > 1/2. 

Thus, the function F(t) can be represented as a sum of elementary 
signals 


F(t) = 


■EE 


Gjnn 9(t; "tm , f n ) 


m 


(4) 
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The locations t m and f n are usually, but not necessarily, regularly 

spaced, e.g. t„, = m t,Af n = — . So long as the sampling density conforms 

At 

to the sampling theorem, i.e., one (two) degree(s) of freedom per cell size 
AtAf = one-half (one), the rectangular cell shapes are arbitrary. 

Equation (4) can be equivalently written for a (two-dimensional) 
spatial location-spatial frequency domain as 


F(x,y) 



g(x,y; Xj|j, Y m ; un» wn) » 


(5) 


where (X m , Y m ) represents the spatial location and ( u n , w n ) the spatial 
frequency of the elementary cell, and 


g(x,y; X m , Y m , u n , w n ) = [2TTa x (m)a y (m)]' 1/2 e" (x ' x m )2/4a x 2(m) " 


(y-y m ) 2 /4cy 2 ( m ) - i 2mj n ( x-x m ) - i 2^ (y-y m ) 


For the most straightforward generation of Gabor complex spectrums with the 
direct electronic Fourier transform (DEFT) device, H>12 we assume all 
elemental cells are of uniform shape, letting o x (m) = °x an d ° y (ro) = 

°y. The elementary cells sufficient to satisfy the generalized sampling 
theorem and, hence, represent an arbitrary function, are AxAu = AyAw = 

1 where we define Ax = 2 Vtto x and Ay = 2V^Oy* The 4-space elemental 
volume is V4 = AxAyAuAw = l. Note that Eq. (5) is a complex expansion 
where there are two degrees of freedom associated with each G mn and the 
entire frequency plane is used. For real functions, F(x,y), it is only 
necessary to cover a frequency half-plane. 
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3. Electroacoustic Imaging Device 


The current I ( t) generated by an electroacoustic imaging device is 
proportional to 


I(t) 



L s (x,y) 


E z 2 (x,y,t), 


( 6 ) 


where E z (x,y,t) = Ei z (x,y,t) + E 2 z (x,y,t) represents the normal component 
of the electric field associated with the two acoustic traveling waves that 
modulate the electronic charges induced by the image radiance field 
L s (x,y), as illustrated in Fig. 3. The two orthogonal acoustic waves are 
generated by the Gaussian modulated sinusoidal signals as suggested by the 
Gabor representation and given by 


s^(t) 

» V je -(t-Tl ) 2 /40i2 

cos 27rfj(t-Tj) 

(7a) 

S2(t) 

= V 2 e -<t-T 2 > 2 /4o 2 2 

cos 2 irf 2 (t-T 2 ) 

(7b) 


for the x and y 
waves are given by 


E lz (x,y,t) = E 


E 2z (x,y,t) = E 


inputs, respectively. The resulting traveling electric 




T 1 ) 2 /4o 1 2 cqs 2rfi ( t _ x__ T i , 

T 2 ) /4a2 2 cqs 27T f 2 (t - -x 2 ) 


(8a) 

(8b) 


where vj, V 2 are the respective surface acoustic wave velocities. 
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To insure that the portion of the integrand of Eq. (6) resulting as 
the coefficient of the desired sensing frequency sinusoid varies slowly 
with respect to the sensing frequency, the sum frequency (f]+f 2 ) term is 
selected and the difference (fj^) and double (2fj, 2 f 2 ) frequency terms 
are discriminated against. The sum frequency component, I+(t), is 

I+(t) = R e Cl+(t) E 1 z E 2z e l2ir(f l + f 2 H ]. (9) 


The phasor signal I+(t) can be written explicitly as a function of 
spatial location (x 0 , y 0 ) and spatial frequency (o,<d) as 


I + (x c , y Q ; u ,(i) ) = e" i(<J) l + <l>2) j(/dxdy L s (x,y) e { " (x " x o )2/4a x 2 -0^) 2 /4 a y 2 


- i2rr(ux + wy)} 


( 10 ) 


where the peak of the Gaussian window pulse that propagates across the 
electroacoustic device is located at x 0 = vj(t - T^) and y 0 = V 2 (t - T 2 ), 
u = f]/vj, oj = f 2 /v 2 » <f>i = 2irfjx^, <j >2 = 2 rrf 2 T 2 » a x = Vjcfi, Oy - ^2P2’ 

For devices that operate in the 100 MHz range and SAW velocities 
4 x 10^ cm/sec, the center spatial frequencies are rather high, of the 
order of f/v = 250 cycles/cm. It is thus necessary to sample the radiance 
field in order to gain access to the lower spatial frequencies of the 
complex spectrogram. This is accomplished by an interdigital contact 
pattern in one direction and etching of the photoconductor layer in the 
other. H>12 Neglecting the finiteness of the sampling array, the spatial 
frequency spectrum of the sampled radiance field is given by 


L S K«) 


6xSy 

XY 




» - E) sine <2f) sine <"^1 , 


(ID 
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where (X,Y) are the sampling intervals, (5x, 6y) are the aperture 

/N 

dimensions and L(u,w) is the spectrum of the original field. The reduced 
spatial frequencies are defined by u Q = u - u 0 = id - For L(x,y) 
band-limited to |u| £ juj £ and (u 0 , restricted to the same 

range, it can be shown that for I+(x 0 , y 0 ; u 0 , w 0 ) = I + (x 0 ,y 0 ; u 0 + 

A 1. 
w o + y 


Vv 


y ;U , 

J o 0 




= p-i^i + 




dxdy L(x,y) e' 


(x-x 0 ) 2 /4a x 2 - (y-y 0 ) 2 /4oy 2 


-i2iT(u 0 x + ai 0 y) 


(12) 


We define the complex Gabor spectrogram by 


G(x 0 , y 0 ; u 0 , w 0 ) =^dxdy L(x,y) g(x,y; x 0 , y 0 , u 0 , w 0 ) (13a) 


Therefore , 


G(Xq , yg » » w o^ 


= (2TTa x a y )" 1/2 e i(( t ) l + t * > 2 ) + i2 *( u o x o + “oV 


I+(x 0 , y 0 ; V w o)- 


(13b) 


The phase factors (<!>i + $ 2 ) and 27T ( u o x o + w o^o^ are > in Principle, 
known. Thus, a measurement of the magnitude and phase of the signal 
current, I+, yields the complex Gabor spectrogram. The location (x 0 ,y 0 ) 
of the peak of the intersecting Gaussian waveforms follows the trajectory 

*0 + V T 1 - V (14 > 

\ Vj/ 


as illustrated in Fig 3. The Gabor receptive field for a fixed frequency 

\ 

(u 0 ,w 0 ) is moved for that frequency across the entire x,y image plane by 
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varying the delays Tj, Tg. The frequency is then stepped to the adjacent 
spatial frequency cell, and the receptive field is again moved across the 
x,y image; and so forth, until the entire spatial -spatial frequency domain 
has been covered. 

The complex spectrogram and the desired coefficients G mn in Eq. (5) 
are completely specified by the value of the complex spectrogram on the 
Gabor lattice.^ It is sufficient then to sample the signal current along 
each scan line at time intervals corresponding through the wave velocities 
to the Gabor lattice. Although there is some overlap of the Gabor 
functions, the coefficients G mn can be approximated by these sampled 
values of signal current, that is 



It is interesting to observe that the electroacoustic imaging device 
could be operated as a conventional imaging system (sum frequency mode) or 
a Fourier transform system (difference frequency mode) simply by adjusting 
the standard deviation a of the Gaussian envelope. As a ^ 0, the 
elementary signal becomes a delta function and the device functions as an 
imaging system; and as o -*■ the elementary signal becomes a sinusoid and 
the device functions as a direct electronic Fourier transform (DEFT) 
system. Other types of complex spectrograms can also be obtained. For 
example, rather than the Gaussian windows, a rectangular window function of 
size AxAy may be applied. In this case, the sampled signal current , 
values are exactly the desired expansion coefficients of the associated 
"Gabor type" expansion. 
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Concluding Remarks 

It would be difficult to predict the ultimate impact of this 
approach--that is, of the sensing, characterization, and recognition of 
elementary signal s--on artificial vision. 

Clearly, however, one should not be apprehensive because it initially 
may appear to be rather abstract and perhaps even alien to our intuitive 
perception of our own visual processes. This approach is, as Gabor's aptly 
phrased designation "elementary signal" suggests, deeply rooted in 
mathematics concerned with characterizing natural phenomena. 

Perhaps it would be more appropriate to regard this approach merely as 
another transform for encoding signals. As such, it might be expected to 
have certain advantages for encoding spatial features and patterns for 
subsequent autonomous characterization and recognition. Our expectation 
arises primarily from two observations: one, some recent efforts to model 

human visual processes are based on Gabor's concept of elementary signals 
as the basic visual information-carrying signals from eye to brain: and, 

two, a recent laboratory realization of elementary signal representation of 
spoken words indicates that this type of representation is particularly 
suitable for generating recognizable speech patterns. 
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FIG. 1. GABOR REPRESENTATION OF A MODEL PROPOSED BY SAKITT AND BARLOWlO FOR 
THE FIRST STAGE OF THE CORTICAL TRANSFORMATION OF VISUAL IMAGES. THE 
ARRANGEMENT OF CELLS FOR ENCODING TWO-DIMENSIONAL IMAGES IS DISCUSSED 
IN REF. 10. 
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(a) Gaussian modulated sinusoidal signal. (b) Scan pattern of Gaussian modulated 

sinusoidal signal. 


FIG. 3. OPERATION OF ELECTROACOUSTIC DEVICE TO OBTAIN ELEMENTARY SIGNAL 
REPRESENTATIONS OF INCOHERENT RADIANCE FIELDS. 
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